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ABSTRACT 

The  theory  of  compressed  sensing  has  shown  that  sparse  sig¬ 
nals  can  be  reconstructed  exactly  from  remarkably  few  mea¬ 
surements.  In  this  paper  we  consider  a  nonconvex  extension, 
where  the  norm  of  the  basis  pursuit  algorithm  is  replaced 
with  the  norm,  for  p  <  1 .  In  the  context  of  sparse  error 
correction,  we  perform  numerical  experiments  that  show  that 
for  a  fixed  number  of  measurements,  errors  of  larger  support 
can  be  corrected  in  the  nonconvex  case.  We  also  provide  a 
theoretical  justification  for  why  this  should  be  so. 

Index  Terms —  Signal  reconstruction,  error  correction, 
minimization  methods,  linear  codes,  random  codes. 

1.  INTRODUCTION 

Recent  papers  [1,  2]  have  introduced  the  concept  of  com¬ 
pressed  sensing.  The  basic  principle  is  that  sparse  or  com¬ 
pressible  signals  can  be  reconstructed  from  a  limited  (or  com¬ 
pressed)  number  of  random  projections.  A  few  of  the  many 
potential  applications  are  medical  image  reconstruction  [3], 
image  acquisition  [4],  and  sensor  networks  [5]. 

The  first  algorithm  presented  in  this  context  is  known  as 
basis  pursuit  [6].  Let  <!>  be  an  M  x  N  measurement  matrix, 
and  <!>/  the  vector  of  M  measurements  of  an  N-dimensional 
signal  /.  The  reconstructed  signal  u*  is  the  minimizer  of  the 
£^  norm,  subject  to  the  data; 

min||tt||i,  subject  to  <i)u  =  <!>/.  (1) 

U 

A  remarkable  result  from  [7]  is  that  if  the  rows  of  $  are 
randomly  chosen,  standard-normally  distributed  vectors,  there 
is  a  constant  C  such  that  if  the  support  of  /  has  size  K  and 
M  >  CKlog{K/N),  then  the  solution  to  (1)  will  be  exactly 
u*  =  f  with  overwhelming  probability.  The  required  C  de¬ 
pends  on  the  desired  probability  of  success,  which  in  any  case 
tends  to  one  as  iV  ^  oo. 

Variants  of  this  result  include  <!>  being  a  random  Fourier 
submatrix,  or  having  values  ±\/sfN  with  equal  probability. 
Also,  /  can  be  sparse  with  respect  to  any  basis,  with  u  re¬ 
placed  with  vku  for  suitable  unitary  4'. 

A  family  of  iterative  greedy  algorithms  [8,  9,  10]  have 
been  shown  to  enjoy  a  similar  exact  reconstruction  property. 
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generally  with  less  computational  complexity.  However,  these 
algorithms  require  more  measurements  for  exact  reconstruc¬ 
tion  than  the  basis  pursuit  method. 

In  this  paper,  we  take  the  opposite  approach,  and  show 
that  a  nonconvex  variant  of  basis  pursuit  will  produce  exact 
reconstruction  with  fewer  measurements.  Specifically,  we  re¬ 
place  the  £^  norm  with  the  norm,  where  0  <  p  <  1  (in 
which  case  ||  •  ||j,  isn’t  actually  a  norm,  though  d{x,y)  = 
||a;  —  y\\^  is  a  metric); 

min||tt||P  subject  to  <I>u  =  <!>/.  (2) 

That  fewer  measurements  are  required  for  exact  reconstruc¬ 
tion  than  when  p  =  1  was  demonstrated  by  numerical  ex¬ 
periments  in  [11],  with  random  and  nonrandom  Fourier  mea¬ 
surements.  A  similar  approach  was  used  by  Rao  and  Kreutz- 
Delgado  [12]  for  basis  selection.  In  this  paper,  we  consider 
the  context  of  error  correction,  and  our  measurements  will  be 
random  Gaussian  projections.  In  Section  2  we  provide  a  theo¬ 
retical  result  (based  on  one  from  [13])  justifying  the  increased 
likelihood  of  exact  reconstruction.  In  Section  3,  numerical  ex¬ 
periments  will  show  that  using  p  <  1  allows  perfect  recovery 
from  the  corruption  of  a  greater  number  of  entries. 

2.  ERROR  CORRECTION 

We  consider  the  abstract  encryption  framework  described  in 
[13].  Let  A  be  an  m  X  n  matrix,  with  rn  >  n.  If  A  has  full 
rank,  we  can  regard  it  as  a  linear  block  cipher,  with  a  plaintext 
/  G  M"  encrypted  as  A/.  We  suppose  the  ciphertext  A/  is 
corrupted  by  an  error  vector  e  G  M™,  with  the  property  that 
the  support  of  e  is  at  most  r;  ||e||o  <  r.  Given  the  corrupted 
ciphertext  y  =  Af  e,  under  what  circumstances  can  we 
recover  Af  (and  hence  /)  exactly? 

This  problem  can  be  recast  into  the  form  of  (2)  by  the  use 
of  a  matrix  B  whose  kernel  is  the  range  of  A.  Then  By  = 
B{Af  e)  =  Be.  We  attempt  to  reconstruct  e  from  the 
measurement  vector  By(=  Be),  by  solving  (2); 

min||(i||P  subject  to  Bd=Be.  (3) 

d  ^ 

If  the  unique  minimizer  is  d  =  e,  then  we  will  have  the  error 
vector  e,  from  which  we  can  recover  the  plaintext  from  Af  = 
y-e. 
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Bu  =  Be  Bu  =  Be 


(a)  p  <  1  (b)  p  =  1 


Fig.  1.  Exact  reconstruction  occurs  if  the  constraint  plane 
meets  the  £p  sphere  containing  e  only  at  e.  For  sparse  e,  this 
condition  is  more  likely  for  p  <  1  if  one  only  need  consider 
points  near  e. 

Still  following  [13],  the  suhstitution  d  =  y  —  Af  yields 
the  unconstrained  problem 

mm\\y-Af\\P,  (4) 

/ 

as  B{y  —  Af)  =  By  =  Be  for  all  /  G  M".  Our  numeri¬ 
cal  experiments  in  Section  3  will  consist  of  solving  (4)  and 
comparing  the  minimizer  with  /. 

The  geometry  of  (3)  is  depicted  in  Figure  1.  Exact  recon¬ 
struction  corresponds  to  e  being  the  only  point  of  intersection 
of  the  affine  space  Bd  =  Be  and  the  f^’-sphere  containing  e. 
If  e  is  sparse,  this  will  be  true  for  many  B,  seemingly  to  the 
same  degree  whether  p  =  1  orp  <  1.  This  changes,  however, 
in  higher  dimensions,  or  if  a  solution  of  (3)  must  also  be  close 
to  e.  The  smaller  p  is,  and  the  closer  a  solution  to  (3)  must  be 
to  e,  the  more  likely  that  a  given  choice  of  B  will  yield  exact 
reconstruction.  And  it  is  the  sparsity  of  e  that  will  contribute 
to  the  requirement  that  a  minimizer  be  close  to  e. 

This  brings  us  to  the  concept  of  an  approximate  S- 
restricted  isometry,  as  introduced  in  [14].  For  ak  x  m  matrix 
B  and  T  C  m},  let  Bt  be  the  matrix  consisting  of 

the  columns  bj  of  B  for  j  €  T.  (We  will  use  similar  notation 
for  vectors,  with  urit)  =  u{t)  if  f  €  T  and  0  otherwise.)  For 
each  number  S,  define  the  ^-restricted  isometry  constant  of 
B  to  be  the  smallest  (5s  >  0  such  that  for  all  subsets  T  with 
\T\  <  S' and  all  c  € 

{l-Ss)\\cg<\\BTc\\l<{l  +  6s)\\c\\l.  (5) 

Thus  if  To  is  the  support  of  e,  Bd  =  Be,  and  d  is  supported 
on  To,  we  will  have  ||d  —  e|||  <  \\B{d  —  e)|||/(l  —  6r)  = 
0,  provided  Sr  <  1-  However,  there  is  no  guarantee  that  a 
minimizer  of  (3)  will  be  supported  on  Tq,  or  even  be  sparse. 

Working  in  tandem  with  (5)  will  be  the  following  obser¬ 
vation,  essentially  from  [15].  Fet  dhe  a  solution  of  (3),  and 
let  h  =  d  —  e.  By  the  triangle  inequality  for  ||  •  ||^,  we  have 

ll|e||^-||-/tToll^|<||e  +  fiTol|^.  (6) 


Since  Tg  n  Tg  =  0,  we  have 

l|e||p  -  WhToWp  +  l|/iTo“|lp  <  l|e  +  ^To  +  hr^Wp 

=  \\e  +  hrp  =  \\d\\P<\\e\\P, 

the  last  inequality  holding  because  d  solves  (3).  The  result  is 
that 

WhTsK^WhToK-  (8) 

In  other  words,  although  d  need  not  be  sparse,  a  bound  exists 
on  the  portion  of  d  outside  the  support  of  e  (note  that  dxg  = 
hx^).  The  more  sparse  e  is,  the  stronger  (8)  is. 

The  final  piece  of  this  picture  is  the  following  result.  It 
quantifies  the  restricted  isometry  condition  necessary  for  ex¬ 
act  reconstmction,  and  generalizes  and  improves  for  p  <  1 
the  corresponding  result  of  [13]. 

Theorem  2.1.  Let  the  block  cipher  A  be  an  m  x  n  matrix 
with  m  >  n.  Let  f  G  K"  be  a  plaintext,  e  G  an  error 
vector,  and  let  r  =  ||e||o  fie  the  size  of  the  support  of  e.  Let 
B  be  a  matrix  whose  kernel  is  the  range  of  A.  Let  p  G  (0, 1], 
a  =  Suppose  that  B  satisfies 

Sar  +  35(a+l)r  <  2.  (9) 

Then  the  unique  minimizer  of  (3)  is  exactly  e,  and  we  can 
recover  the  plaintext  f  exactly  from  the  corrupted  ciphertext 
y  =  Af  +  e  as  the  unique  minimizer  of  (4). 

Forp  =  1,  this  is  exactly  as  appears  in  [13].  For  a  given 
B,  the  restricted  isometry  condition  (9)  will  hold  for  larger 
values  of  r  when  p  <  1.  We  thus  can  expect  to  be  able  to 
correct  errors  of  larger  support  in  this  case. 

It  is  also  shown  in  [13]  that  in  the  case  of  random,  Gaus¬ 
sian  ciphers,  the  condition  of  Theorem  2.1  holds  (for  p  =  1, 
a  fortiori  for  p  <  1)  with  overwhelming  probability,  provided 
r  <  pm  for  some  constant  p.  The  value  of  p  given  is  very  far 
from  sharp,  however. 


Proof  of  Theorem  2.1.  The  proof  generally  follows  the  lines 
of  [13],  but  with  a  simplification.  (Specifically,  equation  (2.2) 
therein  is  not  required.)  As  above,  we  consider  a  solution  d  of 
(3)  (that  such  exists  is  geometrically  obvious).  Fet  h  =  d  —  e', 
we  wish  to  show  that  h  =  0.  Fet  Tq  be  the  support  of  e.  Fet 
M  =  ar.  Arrange  the  elements  of  T§  in  order  of  decreasing 
magnitude  of  |fi,|  and  partition  into  T§  =  Ti  \J  T2  U  ■  ■  ■  U  Tl, 
where  each  Tj  has  M  elements  (except  possibly  T[f).  We  do 
this  because  the  restricted  isometry  condition  gives  us  control 
over  the  action  of  B  on  small  sets.  Denote  Tqi  =  Tq  U  Ti. 
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We  decompose  Bh: 


3.  NUMERICAL  EXPERIMENTS 


0=\\Bd-Be\\2  =  \\Bh\\2  = 


Btoi  hxoi  +  'Yh 

j=2 


>  ||Stoi/iToi||2  - 


i=2 

L 


>  ||Stoi/iToi||2  -  Y  W^TjhTjh 


j=2 


>  a/I  —  SM+r\\hToi  II2  —  V'l  +  (5m  \\hTj  ||2- 

i=2 


(10) 


Now  we  need  to  control  the  size  of  the  ||/iTj  lb-  We  aim  to  use 
(8),  for  which  we  must  estimate  the  norm  in  terms  of  the 
£P  norm.  For  each  t  E  Tj  and  s  G  7}-i,  \h{t)\  <  |/i(s)|,  so 
that 

\Ht)\p<\\hT^_X/M.  (11) 

Then 

(12) 

so  that 

Eiiv,ii2<(i;iiv,  lib 

j=2  S  =  1  ^  (^-’1 

where  we  have  used  the  reverse  triangle  inequality  property 
of  the  £P  norm  for  p  <  1.  Now  we  may  use  (8),  and  then 
convert  back  from  £p  to  by  means  of  Holder’s  inequality; 

p  1—2 

=  (Ei^Wn  (Ei) 

teTo  V  To  /  V  To  / 

=  l|(iTolb|Tob-^/b 

(14) 


Combining,  we  obtain 


Y  -  II^Tollp/Af^^^  <  WhTohO-^'^ 

j=2  ^  2' 

=  II  (iTo  II 2/^/3. 


i  _  i 
p  2 


(15) 


Putting  together  with  (10),  we  have 

o>^/r^  ^M+r  l|(lToill2  -  \/l  +  (5m||(iToI|2/V3 

>  (^i/l  —  ^M+r  —  a/I  +  (JM/vblj  ll^lToilb- 

The  condition  (9)  of  the  theorem  ensures  that  the  scalar  factor 
is  positive,  so  (itoi  =  0.  In  particular,  Hto  =  0;  then  h  =  0 
follows  from  (8).  □ 


We  present  the  results  of  numerical  experiments  investigating 
the  ability  of  (4)  to  reconstruct  a  plaintext  from  a  corrupted 
ciphertext.  We  adopt  the  approach  of  [13],  to  facilitate  di¬ 
rect  comparison.  We  used  n  =  128,  and  both  m  =  256  and 
m  =  512.  For  each  m,  we  used  20  different  values  of  r, 
chosen  as  a  percentage  of  m.  For  each  value  of  m  and  r, 
the  following  was  repeated  100  times.  The  elements  of  the 
m  X  n  cipher  A  and  the  n  x  1  plaintext  /  were  randomly 
chosen  from  the  standard  normal  distribution.  The  r  entries 
to  be  corrupted  were  randomly  chosen,  and  the  correspond¬ 
ing  values  of  the  error  vector  e  were  chosen  from  the  standard 
normal  distribution.  We  let  y  =  Af  +  e,  and  computed  a  local 
minimizer  /*  of  (4),  for  each  p  G  {0.1, 0.2, . . . ,  1}.  The  re¬ 
construction  was  deemed  exact  if  every  entry  of  |/*  —  f\  was 
less  than  10“®;  forp  <  1,  such  “exact”  maximum  residuals 
were  generally  less  than  10“^®.  Further  iteration  of  the  algo¬ 
rithm  below  would  generally  reduce  p  =  1  residuals  below 
10“^®  as  well. 

To  compute  a  local  minimizer  of  (4),  we  used  an  algorithm 
based  on  the  lagged-diffusivity  algorithm  of  Vogel  and  Oman 
[16]  for  total-variation  minimization.  Consider  the  Euler- 
Lagrange  equation  for  (4); 

A^\Af-yr^{Af-y)  =  0.  (17) 

Given  the  nth  iterate  /„,  we  solve  for  the  next  iterate  /„+i 
by  “lagging”  the  nonlinear  terms  in  (17),  resulting  in  a  linear 
equation; 

A^\AU  -  yr^AX^  =  A^\AU  -  yf-^y.  (18) 

The  iteration  was  begun  with  the  least-squares  solution  (that 
forp  =  2).  To  avoid  division  by  zero,  \Af  —  y  \  was  approx- 
imated  by  ((A/  —  j/)^  +  e)  .  The  value  of  e  was  initially 
set  to  1,  and  the  minimizer  computed.  The  process  was  then 
iterated  with  e  100  times  smaller  than  the  previous  value,  and 
with  the  previous  minimizer  used  as  the  initial  iterate,  a  total 
of  10  times.  The  entire  process  took  approximately  9  seconds 
on  a  2.8  GHz  processor  for  m  =  512,  3  seconds  for  m  =  256. 

Results  of  the  experiments  are  plotted  in  Figure  2.  Call 
the  corruption  rate  p  =  r/m.  For  plaintext  size  n  =  256 
and  p  =  1,  exact  reconstruction  occurred  all  100  times  for  a 
corruption  rate  of  p  <  10%,  and  99  times  for  p  =  15%.  Using 
p  =  0.9  gave  exact  reconstruction  100  times  for  p  <  15% 
and  99  times  for  p  =  17.5%.  For  p  =  0.8  or  less,  exact 
reconstruction  always  occurred  for  p  <  20%. 

When  the  plaintext  size  was  n  =  512,  exact  reconstruc¬ 
tion  occurred  always  for  p  =  1  when  p  <  32.5%,  99%  of 
the  time  for  p  =  35%.  For  p  =  0.9,  we  had  100%  exact 
reconstruction  for  p  <  40%,  and  99  times  for  p  =  42.5%. 
Decreasing  p  to  0.8  or  0.7  increased  the  corresponding  values 
of  p  to  42.5%  and  45%.  For  p  =  0.6  and  0.5,  exact  recon¬ 
struction  always  occurred  for  p  <  45%.  For  p  <  0.4,  this 
happened  for  p  <  47.5%. 
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(a)  m  =  256 


(b)  m  =  512 

Fig.  2.  Plots  of  observed  frequency  of  exact  reconstruction 
versus  sparsity  of  ciphertext  errors,  for  values  of  p  used  in 
minimization  ranging  from  1  (solid  line,  square  marker)  down 
to  0.1  (solid  line,  ‘+’  marker;  values  from  0.9  to  0.2  are  dotted 
lines,  from  left  to  right).  Even  p  =  0.9  shows  substantial 
improvement  overp  =  1.  When  m,  the  ciphertext  size,  is  256, 
decreasing  p  from  1  to  0.8  or  lower  allows  an  additional  25 
entries  to  be  corrupted  and  still  expect  exact  reconstmction  of 
the  plaintext.  For  m  =  512,  77  more  entries  can  be  cormpted 
by  decreasing  p  to  0.4  or  lower. 

Considering  all  observed  probabilities  of  exact  reconstmc¬ 
tion,  from  the  plots  we  see  that  even  a  decrease  of  p  from  1  to 
0.9  results  in  a  substantial  improvement.  Decreasing  p  further 
yields  improvement,  but  by  less  and  less  as  p  gets  smaller. 
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